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ABSTRACT 

This status report discusses the issues and concerns 
of assessing, at a national level, the ability of college graduates 
to think critically, solve problems, and communicate effectively 
(Nat ional Education Goal 5) • Two workshops have been held on the 
issue and participants indicated that assessment requires a working 
consensus on the specific skills to be measured and must be 
considered from the perspective of the teaching/learning experience 
as a whole. The report notes that a delphi study is planned involving 
faculty, employers, policymakers, and others which will attempt to 
develop a working consensus concerning workplace and citizenship 
skill needs of the larger community. This study will be followed by a 
more formal process to design a process to assess the teaching and 
learning of these skills. Activities to date have found that the 
greatest concern is not whether skills and abilities are identified 
and validated, but how they should be assessed. From the 
institutional perspective the concern is that assessments focusing on 
workplace and citizenship skills may cause a reshaping of the role of 
many colleges, and that development of workplace/citizenship skills 
may sometimes be inconsistent with the individual freedoms of 
students and faculty. Concluding comments argue that, based on the 
reasons given by students for attending college, the identification 
and assessment of workplace/citizenship skills is relevant in today's 
world. (Contains four references.) (GLR) 
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National Assessment of College Student Learning: A Status Report 
^ Sal Corrallo 

February 1, 1993 

v© 

O The November-December 1991 issue of Assessment Update 

reported that the National Center for Education Statistics was 
about to undertake an effort to identify ways of measuring the 
ability of college graduates to think critically, solve problems, 
and communicate effectively. The project was undertaken in direct 
support of National Education Goal 5. Goal 5, was conceived in 
response to a concern that the nation's ability to compete in a 
global economy was hampered by a lack of education and training by, 
among others, large numbers of postsecondary graduates. After 
considering the barriers or problems standing in the way of 
achieving the goal, five objectives were identified for attention 
and as necessary first steps. The fifth objective focused on the 
higher order thinking and communication skills of college 
graduates. These skills were to be thought of in terms of their 
utility in the workplace and for the practice of citizenship. 
Since that time NCES has conducted two planning workshops, the 
first in November 1991. Participants were asked to consider the 
issues and concerns for each stage of the project, i.e., what 
should be assessed, at what level (s) of achievement, when and how? 
The commissioned background papers, reviews and workshop 
N discussions, covered all four issues, in varying degrees. Most of 



0 

ERIC 



the discussion focused on the application of assessment process. 
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Notwithstanding, it was clear from the papers and the proceedings 
that until a working consensus of the higher order thinking and 
communication skills and related levels of assessment is compiled 
and consideration giv as to how these skills may be measured, it 
is futile to discuss the time and means of assessment. The 1992 
National Education Goals Panel Report indicates that "We need a 
clearer understanding of the knowledge and skills. . .graduates 
attain and how they relate to the demands of the world marketplace 
and the rights and responsibilities of citizenship," 1 Works! - p 
participants were in general agreement that assessment for 
accountability, alone, would not serve the purposes of the National 
Goal. Assessment must be considered from the perspective of the 
teaching/learning experience as a whole. 

Skills identification was the primary theme of the second workshop, 
conducted in November 1992. Four papers were commissioned to 
identify skills, levels of achievement and examples of how these 
skills might be assessed. These included single papers on Speaking 
and Listening and Reading and Writing along with two papers on 
Critical Thinking and Problem Solving. Each paper was used as the 
basis of one of four workshops. Representatives, primarily from 
the business community and college faculty, attempted to reach some 
level of closure on the specific sets of speaking and listening, 
reading and writing, and problem solving and critical thinking 
skills a college graduate should master prior to graduation. The 
results were mixed. Some work group participants, primarily 
faculty members, were still coming to grips with the notion of 
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national assessment of college student learning. In essence they, 
like many in the first workshop, argued that the learning 
experience is so personal that the only effective way to assess is 
to observe individual student learning over an extended period of 
time. As might be expected workshop participants provided a mixed 
set of suggestions. These included listings of problem solving and 
critical thinking skills, a listing of speaking and listening 
skills, and a rationale for the assessment of reading and writing 
skills. The papers, reviews and proceedings will be published 
during the summer of 1993. 

In 1993, in a continuing effort to identify, and then develop a 
working consensus of the workplace and citizenship skill needs of 
the larger community, NCES contracted with the Center for the Study 
of Higher Education at Penn State University. They have been asked 
to conduct a two step delphi survey, using the materials developed 
for the second workshop but supplemented by information from other 
sources. They will also use an extensive group of advisors to 
develop the initial listing of skills. The delphi survey group 
will include 270 faculty, 140 employers, 100 policymakers, and 80 
persons who participated in the earlier workshops. Of critical 
importance are the opinions of faculty and members of the workplace 
community. The final report, which will published, will summarize 
the results of the survey for each of the skill areas. In addition 
the National Center for Higher Education Management Systems has 
been given a task order to inventory and evaluate current indirect 
approaches to assessment at the postsecondary level. The findings 
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from each of these efforts have or will be published to assure that 
the larger community is kept informed. 

All of these efforts are considered preliminary to a more formal 
process to identify and design a process to assess the teaching 
learning of these skills. To this end a study design contractor 
(Phase I) will be selected by the end of the September 1993. The 
study design contractor will, drawing upon all previous NCES work 
and other sources, be responsible for the development of a set of 
analytical frameworks for each of the skill areas. Each framework 
will include the set of skills to be assessed, scales for assessing 
levels of achievement, and examples of alternative approaches for 
assessment • The frameworks will be used as the basis for the 
design and implementation of the formal assessment process (Phase 
II) . This contract will begin in 1996 and extend for 4 years. 

So much for the mechanics of the process. What has been learned to 
date, regarding the feasibility and desirability of the project? 
The greatest concern in both workshops, and in much of the 
literature, is not whether there are sets of work and citizenship 
skills and abilities that may be identified and validated, but 
rather how these should be assessed? First it is feared that a set 
of higher order thinking and communication skills will be 
identified and the scales used to measure competency will become 
the basis for the development of standards of college student 
learning. Second most of the assessment instrumentation used has 
been developed not by the academic or business community, but by 
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the' commercial testing industry with standards or norms developed 
outside the academic or workplace community. Third, there is 
concern that the focus upon general intellectual skills such as 
problem solving, critical thinking, and communication skills is too 
narrow and that content will be lost. Fourth teaching to the test, 
both in terms of form and content, can narrow rather than broaden 
the college learning experience. Fifth attention will not be given 
to identifying skills that the college graduate should master 
before they leave college, other than those needed for the 
workplace and citizenship. 

From an institutional perspective, were the assessment to focus 
only on the set of workplace and citizenship skills, there is 
concern this would reshape or refocus the role of many colleges. 
Further in higher education, certainly at the four year liberal 
arts colleges and in the universities, there is more implied 
emphasis on individual freedoms for faculty anJ students at 
academic, political, and social levels. Some feel the 

teaching/learning of the skills requirements of , the workplace and 
for citizenship may sometimes be inconsistent with the individual 
freedoms of students and faculty and the attributes of the well 
rounded college graduate. Also some are concerned that 
course/program assessment could result in national standards. They 
point to programs such as accounting, nursing, and pharmacy among 
others, which require graduates to take certification examinations 
upon leaving school. 
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At the end of the second workshop participants were asked to 
identify next steps, potential problems, additional research and 
individuals or groups that needed to be involved* The responses 
were not unexpected. They included: further refinement of the 
skills and competencies to be assessed; the need to link skills 
across disciplines and competencies; validation of the skills both 
from the perspective of work and citizenship; the need to keep the 
project open to all affected publics; and a concern that the 
"national" versus "individualized" nature of assessment be 
recognized and accounted for in developing an assessment approach, 

A closing thought with regard to the means of assessment. As 

suggested earlier, identification of the most popular reason 

2 

freshman give for attending college is to get a better job. 

Further, roughly two-thirds of recent college graduates completed 

3 

a professional/occupational related program. Given the 

enrollment in career related programs and the student's stated 

reason for attending college, doesn't it seem reasonable for 

colleges in general and faculty in particular, to do all they can 

to help the students obtain the knowledge and skills they r.eed to 

cope? Past -studies attempting to link grade point averages with 

job performance found little relationship with the skills and 

competencies individual students need for most jobs. However when 

grades in job related courses are considered, a positive 

4 

relationship with job performance was identified. Thus it 

appears the college graduate is better judged, not by the grades, 
but rather by the skills they bring with them to the job. Relating 
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job performance to skills has implication for those concerned with 
the means. For example if it can be established that mastery of 
specific course content, given that it reflects what students need 
to know for work or citizenship, then classroom grades might be an 
appropriate assessment of the knowledge and skills of all or 
selected groups of college graduates. The issue then would be one 
of certification; are colleges teaching and then assessing 
attainment of the necessary knowledge and skills? 

These concerns will be the primary focus of the major expenditure 
of time, effort and resources over the next thirty months. It is 
an effort that is of vital importance to all who are participants, 
practitioners, or users of postsecondary education. 
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